The present invention relates to image processing and, more particularly, to a device for and a method of modulation of a dynamic range of an image.
Digital images may contain a huge amount of data, especially for high quality display and printing. Colored images are commonly displayed by three or more channels, such as Red-Green-Blue (RGB) channels (for display) or Cyan-Magenta-Yellow-Black (CMYK) images (for print). The source of most images is an imaging device, such as a digital camera, a scanner and the like, in which signal is first captured by a light-sensitive device, and digitized thereafter.
In video capture, digital photography and digital scanning, the image is acquired by means of an electronic sensor such as a charge-coupled device (CCD) or a complimentary metal-oxide semiconductor (CMOS) device with cells sensitive to electromagnetic radiation such as visible light, infrared or ultra-violet waves. In medical or astronomical imaging, images may also be acquired by X-ray, sound-wave, magnetic-field, microwave or radiowave sensitive detectors.
Commercially available digital imaging devices based upon CCD detector arrays are known to acquire image information across a wide dynamic range of the order of 2 to 3 orders of magnitude. It is expected that with the rapid technologically development in the field of digital imaging, this range will most likely be broadened in the near future. Typically however, although at the time of image capture the acquired dynamic range is rather large, a substantial portion of it is lost once the image is digitized, printed or displayed. For example, most images are digitized to 8-bits (256 levels) per color-band, i.e., a dynamic range of about two orders of magnitude. The problem is aggravated once the image is transferred to a display or a print medium which is often limited to about 50 levels per color-band.
A novel imaging technology, recently developed, employs CMOS with active pixel sensors [O. Yadid-Pecht and E. Fossum, “Image Sensor With Ultra-High-Linear-Dynamic Range Utilizing Dual Output CMOS Active Pixel Sensors”, IEEE Trans. Elec. Dev., Special issue on solid state image sensors, Vol. 44, No. 10, 1721-1724], which are capable of locally adjusting the dynamical range, hence to provide a high quality image with high dynamic range.
In addition, over the past years software solutions were developed for fuse multiple exposures of the same scene at low dynamic range (e.g., 256 levels per color-band) into one high dynamic range image (of about 4 orders of magnitudes). Yet, it is recognized that these solutions are only partially exploited due to the bottle neck of low dynamic range during display.
The motivation for developing imaging devices capable of capturing high dynamic range images is explained by the enormous gap between the performances of the presently available devices and the ability of the human visual system to acquire detailed information from an ultra-high dynamic range scene. Specifically, the human visual system, which is capable of acquiring a dynamic range of 14 orders of magnitude, can easily recognize objects in natural light having a dynamic range of 12 orders of magnitude.
Still, there is a growing gap between the state-of-the-art imaging devices and display devices. High quality images, obtained either with photographical film or by digital cameras, suffer, once displayed on a screen or printed as a hard copy from loss in clarity of details and colors at extreme light intensities, within shadows, dark regions, extremely bright regions and/or surfaces close to a lightening source. For example, as a single sharp edge in natural scene (e.g., a shaded object in illuminated scene) can reach a dynamic range of 2 orders of magnitudes, presently available display devices may not be able to recognize such an edge. Another severe problem is that in a specific exposure a dark region of the image may be seen while a bright region is over exposed, or vise versa.
One method of dynamic range compression of images is found in R. Fattal et al., “Gradient Domain High Dynamic Range Compression”, Proc. ACM SIGGRAPH, 2002, where large gradients are attenuated and a low gradient display is constructs by solving the Poisson equation on a modified gradient field.
In an additional method, primarily directed at correcting halo artifacts, high contrast edges are detected while the influence of extreme pixels whose intensity variations are above a factor of 5 are removed, to obtain a dynamic range without the halo artifacts [Pattanaik et al., Proc. SCCG, 24-27, 2002].
The above methods were applied solely on static images (still photography) and shown only limited results, both in terms of the dynamic range being compressed and in terms of the quality of the produced images.
The rational behind the above methods was primarily of mathematical or physical nature. In addition, there are also several methods for compression of a dynamic range of an image, which are based on psychophysiological findings. However, there is no presently known method which is based on physiological mechanisms, such as adaptation.
It is commonly believed that the ability of the human visual system to acquire wide range of illuminations in the same scene is through physiological phenomena known as lightness constancy and lightness gain control. Physiological findings have shown [O. D. Creutzfeldt et al., “The Neurophysiological Correlates of Color and Brightness Contrast in Lateral Geniculate Neurons: I. Population Analysis, and II. Adaptation and Surround Effects”, Exp. Brain Res., 87:1-21, 22-45, 1991] that the induction phenomenon is originated in the retinal receptors and ganglion cells, where in addition to central receptors in the receptive field, surrounding and remote regions of receptors, are inputted to the ganglion cells. Additionally it is hypothesized that the peripheral area that extends far beyond the borders of the classical receptive field of the ganglion cells is also inputted to the ganglion cells, thereby affecting the perceived image.
Application of human vision theories to image processing, which is based on the physiological concept of a center/surround retinal cells, has been attempted in the prior art. For example, to mimic the dynamic range compression of human vision, a detector array with integrated processing in analog silicon chips used a logarithm transformation prior to the surround formation [C. Mead, “Analog VLSI and Neural Systems”, Addison-Wesley, Reading, Mass., 1989].
Another model, commonly known as the Retinex model, is disclosed in U.S. Pat. No. 5,991,456. This method is directed at improving a digital image on an RGB scale both in terms of a dynamic range compression and in terms of color independence from the spectral distribution of the scene illuminant, by subtracting logarithms of intensity values, so as to adjust the intensity of a specific pixel using the surrounding pixels (see also, Jobson et al., “A Multiscale Retinex for Bridging the Gap Between Color Images and the Human Observation of Scenes”, published in IEEE Trans. Image Processing 6:965-976, 1997; and Rahman, Z et al., “A Multi Retinex for Color Rendition and Dynamic Range Compression”, SPIE International Symposium on Optical Science, Engineering, and Instrumentation, conference on Signal and Image Processing).
These attempts, however, fail to address the effect of remote regions of the receptive fields cells (in addition to the color coded retinal cells and physiological adaptation mechanism), hence had only partial success in providing a compression of the huge dynamic range (larger than two order of magnitude) of the scene into the low dynamic range of the display, while maintaining the contrast of the image. It is appreciated that the so-called color constancy mechanism of the visual system, which is related to color coded retinal cells (the cones of the retina) is different than the huge dynamic range compression mechanism, which is related to intensity levels of the rods and cones of the retina. Hence, algorithms attempting to combine color constancy with huge dynamic range (e.g., the Retinex algorithm) unavoidably impose some level of compromising on the physiological assumptions and fail to provide high quality results both for color constancy and for huge dynamic range.
Also of prior art of relevance is International Application No. WO 95/33306, which discloses a signal processor that performs a real time approximation of the non-causal feedback Automatic Gain Control model, for natural process occurring in biological sensors.
In all the prior art image processing techniques, the dynamic range compression is still limited, and images processed using the above methods suffer from non-negligible information loses, once displayed on a conventional display device. In addition, prior art methods fail to provide a physiologically inspired high dynamic range compression in moving images (video).
There is thus a widely recognized need for, and it would be highly advantageous to have, a dynamic range modulation device and a method of modulating an image having a high dynamic range devoid of the above limitations.